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ABSTRACT 

This paper reports on the development and use of a 
multiattribute evaluation model for making resource reallocation 
decisions in a large College of Education. Multiple criteria with 
measurable attributes, procedures for use, and software templates are 
identified, along with data from a recent cycle of reviews. Final 
estimates on weighted utility values for each of the program 
reallocation requests (largely targeted on faculty line items) are 
ranked and illustrated for the decision maker. Eight step-by~step 
processes in multiattribute evaluation techniques are discussed; as 
well as several criteria used (i.e., quality of outcomes, centrality 
to mission, program demand, uniqueness of program, and 
cost-effectiveness), along with their related measurable attributes. 
The study examined specific data concerning the performance of 17 
identified measurable attributes found within five criteria that were 
collected from a 2-year study of 16 alternative reallocation 
requests. The study showed how these 16 alternatives were ranked from 
the most effective and highest valued option, having a utility value 
of 81, to the lowest valued request, having a utility value of 16. 
(Contains 23 references.) (GLR) 
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USING MULTIATTRIBUTE EVALUATION TECHNIQUES IN MAKING 
REALLOCATION DECISIONS IN HIGHER EDUCATION 

Abstract 

This paper reports on the development and use of a 
multiattribute evaluation model for making resource 
reallocation decisions in a large College of 
Education. Multiple criteria with measurable 
attributes, procedures for use, and software 
templates are identified, along with data from a 
recent cycle of reviews. Final estimates on weighted 
utility values for each of the program reallocation 
requests (largely targeted on faculty line items) are 
ranked and illustrated for the decision maker. 

This paper reports on the development and use of a mtdtiattribute evaluation 
model for making resource reallocation decisions in a large research oriented 
College of Education. Multiple criteria with measurable attributes, procedures 
for use, and software templates are identified, along with data from a recent cycle 
of reviews. Final estimates on weighted utility values for each of the program 
reallocation requests are ranked and illustrated for the decision maker. 

The multi-dimensional nature of program and college goals and the multiple 
ntmiber of stakeholders involved in reallocation decisions in higher education 
require the unique methods and procedures of multiattribute decision making 
structures and evaluation techniques. This paper reports on the deployment and 
findings from the use of this technique in an institutional case study. Several 
criteria (i.e., quality of outcomes, centrality to mission, program demand, 
imiqueness of program, and cost-effectiveness), along with their related 
measurable attributes, are defined and reported. Procedures for (a) weighting 
the importance of each criteria and attribute, (b) assigning utility values for 
varying degrees of performance, (c) collecting institutional and program data 
derived from program reallocation requests (largely targeted on faculty line 
items), and (d) conducting the analysis and estimating the final weighted utility 
rankings are illustrated. 

Given the chronic nature of financial problems higher education is expected to 
face during the decade of the 1990s, it is a certainty that almost all institutions will 
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be reviewing and reallocating resources as faculty retire and leave their teaching 
and research positions. The model £ind procedxires illustrated in this study 
should be useful to decision makers as they face these problems. 
The Multiattribute Evffliifiiinn Fram^wnrk 

M^altiattribute utility [MAU] evaluation techniques have been employed within 
the mEuiagement sciences for a nxmiber of years as a means of structuring 
decisions for private sector strategic planning (Carroll & Johnson, 1990; Poole & 
DeSanctis, 1990) and for evaluating program alternatives with miiltiple purposes 
and outcomes (Edwards & Newman, 1982; Keeney & Raiffa, 1976). They have also 
had recent use in assessing and selecting among alternatives in some public 
sector social programs relating to public health (Kaplan, Atkins & Wilson, 1988), 
the criminal justice system (Edwards, 1980), and public education (Levin, 1983; 
Lewis, 1989; Lewis, Johnson, Erickson & Bruininks, 1992). However, in spite of 
many appeals for making rational decisions relative to allocating resources 
within higher education institutions (Hardy, 1988), only two other studies dealing 
with multiattribute evaluation and decision making have been reported in higher 
education. One study reported on the ranking of candidates for tenure decisions 
(McCartt, 1986) and another reported on the selection of a student-information 
system (Blanchard, Pierce & Hood, 1989) using mtdtiattribute evaluation 
techniques. 

As an evaluation and decision making framework, MAU analysis is especially 
appropriate for assisting decision makers in higher education. The multi- 
dimensional nature of goals and the multiple number of stakeholders in higher 
education require the unique methods and procedures of MAU analysis. 
Edwards, Guttentag, and Snapper (1975) have noted that MAU evaluation 
methods and procedures are most appropriately used in settings where (1) the 
evaluations are comparative; (2) programs normally serve multiple 
constituencies; (3) programs normally have multiple goals, not all equally 
important; (4) judgments are a required part of the evaluation; (5) judgments of 
magnitude can be assisted by numerical measurement; and (6) the evaluation is 
relevant to decisions. All of these characteristics are commonly found within 
higher education when decision makers need to identify appropriate outcome 
attributes and make choices about allocating resources between alternative 
programs. 

In simplest terms, a MAU process for decision making and evaluation 
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structures the decision process for a group of stakeholders to make judgments 
about identifying outcomes which measure effectiveness, weighting the 
importance of these indicators, and ranking the alternatives. It requires a 
comparison among two or more alternatives against two or more criteria. These 
criteria also may have several different dimensions (i.e., measurable attributes) 
which need to be identified. After importance weights are attached to each of the 
criteria and attributes, measured dimensions of each attribute are then assigned 
utility values for varying degrees of performance. Based on the measured 
performance of each attribute (either through actual measured performance or 
judgments about performance) within each alternative, utility scores are then 
computed and attached to each of the attributes. These attribute values are then 
multiplied by their importance weights and summed to derive a composite score 
for each alternative. If cost data are available, and if it was not one of the criteria 
or attributes in the evaluation model, cost per vnit of utility can be estimated for 
determining the relative cost-effectiveness or cost-utility (i.e., internal technical 
efficiency) of each alternative. 

The unique quality of a MAU decision and evaluation model is its ability to 
structure the decisions of stakeholders with regard to selecting among 
alternatives with multiple goals, and to frame the evaluation decision into a 
weighted "utility" index for comparison purposes. The term "utility" is used to 
measure the extent to which an alternative satisfies an attribute or criterion. It is 
simply a conventional way of expressing worth, value, or satisfaction in a 
common numerical metric. A MAU model essentially permits an evaluator to 
aggregate the utility or satisfaction derived from each of the various attributes into 
a single measure of the overall utility of the multiattributed alternative. 

MAU procedures may appear to be relatively complex when viewed 
simultaneously; however, when broken down into a step-by-step process the task 
becomes straightforward and fairly simple. There are eight steps in the process 
and include the following activities: 

(1) Identifying the ptirpose and objects of evaluation. 

(2) Identifying relevant stakeholders to assist in making judgments about the 
criteria/attributes and their importance. 

(3) Identifying and organizing the educational criteria and attributes into a 
meaningful structure for analysis. 

(4) Assigning importance weights to the criteria and attributes through 



stakeholder judgments. 

(5) Assigning utility values to the measurement scales for each attribute. 

(6) Collecting measurable performance data on each of the attributes for each 
of the alternatives being evaluated. 

(7) Conducting the MAU technical analysis by aggregating the attribute 
performance measures with their measures of utility and importance for 
each of the alternatives. 

(8) Adding costs, when appropriate, for estimating cost-utility ratios and 
relative efficiency. 

Each of these steps has been described in detail and illustrated for use in other 
settings outside of higher education (see, for example, Edwards & Newman, 1982; 
Lewis et al., 1992). 
Case Study Context and Ptirtxise 

In this case study it was determined that the purpose of the evaluation was to 
assist college decision makers in selecting among alternative programs for the 
expansion, replacement, or retrenchment of program resources. This was 
necessary not only for internal reallocation, but for making decisions about the 
reversion of resources to the central administration of the university. The college 
facing these decisions is a large College of Education located within a large urban 
research university. 

The College of Education of this case study had been undergoing chronic 
financial stress over the past two decades and had recently been required to cut 
additional resoxirces and remit them to central administration. At the same 
time, the college wanted to insure that it maintained program vitality and needed 
to give consideration to reallocating resources to existing and new program 
needs. Short of declaring legal **financial exigency^ and t(irminating current 
staff, the principle sotirce of discretionary resources available to the college for 
retrenchment or reallocation resided in an annual pool of funds that resulted 
from recent faculty and staff resignations, retirements, and deaths. The college 
had been managing a position control system for the past 15 years whereby 
funding for all vacant line items in the budget reverted to the Office of the Dean 
for review and reallocation or retrenchment. 

All data for the case study were drawn from 16 program allocation requests 
submitted by six large departments representing the College of Education and 
incorporating over 48 different programs. Although judgments and decisions 
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about allocations and retrenchments were made annually by the college, for the 
illustrative purposes of this paper all reallocation requests from two years (i.e., 
the 1985-87 fiscal periods) were combined. This combining of reallocation requests 
over two years was done to mask the original rankings and to provide a more 

robust sample for case study illustration.^ 
M^ntifiTtncf the Helevant Stakeholders 

The relevant stakeholders for making judgments about the reallocation of 
resources within the case study college were the Dean of the college and a faculty 
consultative committee composed of elected representatives from the constituent 
departments of the college. All final determinations concerning budget 
allocations within the college were given to the Dean through the authority and 
responsibility vested in the position by the college constitution and imiversity 
policy. Other institutions might have differing institutional arrangements for 
such stakeholder decision making and consultation. 

Tdentifving the Criteria ^Ind Measurable Attributes for ReaUcK^ting Resources 

Most colleges in major universities have mission statements that identify such 
obligations as "to extend continuously the frontiers of knowledge and tested 
skill..., to provide education of high quality..., and to afford leadership in....'' 
Obviously, the criteria for planning, evaluating, and making resource allocation 
decisions for program development within colleges or other academic units must 
relate in some way to such goals. Such goal statements, however, neither state 
nor imply the criteria that are to be used in making judgments relating to 
assessment and resource allocation* Operational criteria that can be used in 
conjunction with these types of goal statements or other statements of priority 
must be developed. Following such development, rational judgments can then be 
made (using quantitative measures of attributes where these are reasonable) to 
determine the degree to which these criteria are met. 

Originally, the identification of criteria for determining an effective program 
within the case study college had been determined during the mid-1970s through 
a joint administration and faculty governance committee of the college over an 
extended two-year period of time (Lewis & Kellogg, 1979). Based upon review of 
existing collegiate statements of mission, planning, and priority, our case study 
college concluded that the major criteria for assessing programs could be 
summarized as quality of outcomes, centrality of mission, program demand, and 

6 




8 



cost-effectiveness. These criteria were subsequently modified with the addition of 
uniqueness of program and adopted by the university as guidehnes for all colleges 
within the imiversity as they faced similar resource issues in the mid-1980s 
(Benjamin, Sauer & Vanselow, 1986). In the all-university guidelines the criteria 
were identified in general terms without reference to their relative importance. 

The identification of criterion indicators was similarly general and only 
suggestive in the all-xmiversity guidelines. Each college was encouraged to use 
whatever data they might have available for making judgments on the identified 
criteria. It was generally assimied that the identification and use of attributes 
would rest upon the judgments of a Dean with consultative assistance from 
faculty. It was believed that the combined evidence based on each of these criteria 
could provide a usefiil profile for assessing a program and for rationally 
establishing priorities for allocating resources to and among the many areas of a 
college or university. It was recognized that the role of these criteria and the 
evidence relating to them would likely vary from one area to another according to 
piirposes and functions. 

The role of these criteria and their measurable attributes take on different 
meanings depending upon the context under which they are being used. During 
expanding or even steady-state financial conditions within higher education, 
many of these general criteria were often employed, in both our case study 
institution and in others, in an informal intuitive manner in making resource 
allocation decisions. In the context of these conditions, most resource allocation 
decisions were viewed as incremental additions to individual programs. Often 
when a senior (i.e., high salaried) faculty member retired or left the university, 
the line item was divided and reallocated to two or more different functions or 
programs. Decision making was at the margin and most often led to the 
expansion of programs. In this context of incremental decision making most of 
the focus was on program quality and enrollment demand. Little serious 
attention was tjrpically given to making hard judgments about centrality, 
imiqueness, or cost-effectiveness (Massy & Wilger, 1992). Obviously, no program 
or group of faculty wanted to be told that their activities had low centrality, 
imiqueness, or cost-effectiveness. Most decisions were driven by student demand 
and enrollments. When low program quality was mentioned, it was largely 
addressed through the counter-argument that a new line item would give the 
program greater capacity to advance its quality. 
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After decades of effortless growth and prosperity, America's postsecondary 
institutions have come under increasing financial stress and waning public 
support. During the first part of the 1980s, this stress largely resulted from a 
slowdown in the economy, with many state legislatures assigning a lower priority 
to higher education than to other social services that included K-12 education. In 
the latter part of the 1980s, overall enrollments in institutions of higher education 
also declined (with concurrent revenue losses), which was compounded with a 
renewed national recession and declining state revenues. At times, these lower 
state revenues, lower priorities to higher education, and lower enrollments have 
resulted in institutional retrenchments. The National Center for Educational 
Statistics (1989, pp. 17-28) projects enrollments to continue falling throughout the 
1990s. Froomkin (1990) has asserted that if revenues in higher education continue 
to depend mainly on the number of students, ""a prolonged depression will be 
experienced by higher education, more serious than anything in the past fifty 
years'" (p. 212). Institutional financial stress is currently with us in higher 
education ^nd will likely continue through the present decade of the 1990s. 

During this past decade budgets were balanced partly at the expense of 
maintenance and repair of buildings and partly at the expense of faculty and staff 
compensation, thus creating additional internal stress within most institutions. 
Unlike expenditure cuts for the maintenance and repair of buildings, lack of 
growth in wage funds resulted in immediate cries of exploitation and despair. 
With the increase in financial troubles and internal strife, higher education also 
faced increasing external demands for greater public accountability along with a 
general loss of public esteem. In response to these calls for greater public 
accountability and the chronic financial stress felt by higher education 
institutions, many of these institutions (especially public imiversities) have today 
begim to get serious about restructuridg and reallocating resources across and 
within their colleges. For the current decade additional criteria relating to 
mission centrality, program uniqueness, and cost-effectiveness have emerged as 
important dimensions for planning, restructuring, and resource reallocation 
decisions. 

Many institutions are no longer asking what needs to be added at the margin to 
make a program stronger or more effective. Rather, they are asking what would 
be lost if the program or activity were no longer being provided by the institution. 
The questions relative to planning and reallocation today are tougher questions 
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than those of the past. They require thinking in terms of opportunity costs rather 
than opportunity gains, and often in terms of bigger issues and greater amoimts 
of resources. 

C:^ia]it:v nf outcomes. The quality of program outcomes must be a primary 
focus of any assessment and reallocation effort. Each activity must be viewed in 
the context of how it contributes to the strength and overall excellence of the 
programs and departments of the collegiate unit. Evidence of the quality of 
departments and programs can come from various sotirces (e.g., rankings by 
external professional associations, faculty publications, participation of faculty in 
professional meetings, special awards and other professional recognition, 
placement of graduate students, average GRE scores of graduate student 
applicants and admissions, number and competitive quality of external grants 
and contracts awarded, peer evaluation, student evaluation, and professional 
constituency evaluation). Distinctions should be made among invited, refereed, 
and other types of professioned contributions; among types of placement; among 
types of recognition; and among levels of significance of research and service 
activities. 

Program evaluations by external bodies such as the American Psychological 
Association and the American Council of Graduate Schools illustrate important 
qualitative estimates of a program's vitality over time. The qualitative 
assessment of a program should include the evaluation of teaching both at the 
graduate and imdergraduate levels. Related evidence includes evaluations by 
students and peers, as well as followup assessments by graduates. Such 
assessment also includes whether the program area has paid special attention to 
the task of consolidating, reorganizing, and otherwise revising the structure and 
content of courses and programs to improve instruction and to serve as a model 
for similar programs in other institutions. Evidence of faculty development 
addressed to meeting new teaching needs should also be reviewed. 

Analogous quality concerns apply for noninstructional program and service 
efforts across a unit. Contributions to professional practice and knowledge are of 
relevance in the qualitative evaluation of professional programs, particularly in a 
professional college. The focus of evaluatingTorltributions to practice in a 
professional college should be on examining the nature and strength of a 
program's relationship to the work of agencies and practitioners in the field. 

From an understanding of these attributes of quaHty, the following measurable 
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attributes of quality were identified by the Dean and faculty consultative 
committee within our case study college: 

• • • Graduate program national ranking 

• • • Quality of faculty publications 

• • • Quality of external fimding 

• • • Quality of instruction 

• • • Quality of professional service 

Centrali< y niiR<f]f^y| Centrality refers to the relative germaneness of an 
activity to the stated programmatic goals and objectives of an orgariizational \mit. 
If one considers the programmatic alternatives that contribute to achieving 
particular goals and purposes, what is the relative necessity of supporting each? 
The critical question to ask is whether the mission (goals and purposes) of a 

, college would be compromised if the program or activity under review were 

diminished or not in existence. In what ways would its absence be noted? If, for 
example, a particular program area were to be eliminated, would this 
compromise the college's national ranking as a professional school in terms of 
scholarly productivity and research, graduate program rankings, and service to 
students, the state, and other constituencies? 

Judgments about the centrality of a program or activity are particularly 
difficult for several reasons. It is a very complex decision and can involve several 
levels of centrality. It is a decision that necessarily rests more on nonquantitative 
or subjective determinations than do the other criteria. It also often rests with 
perceived important political constituencies outside the institution. For these 
reasons a judgment about the centrality of a program or function may be difficult 
to rationalize and act upon. Agreement on the relative centrality of programs 
and activities may be difficult to achieve, especially in the absence of fairly specific 
standards and measures. Nevertheless, the contribution of centrality to resource 

I decision making must be considered. To what degree is the substance of an 

activity pertinent to agreed upon program needs and intentions? In this context, 
two programs might demonstrate the same degree of quality, demand, 
uniqueness, and even cost-effectiveness but might differ in their centrality to 
collegiate programmatic intentions. In a professional college located within a 
large state university, this centrality to '^professional mission and purpose** 
becomes all the more imperative, given its stronger need for focus and rationale 
to preserve it's separate status as a college. 
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Because a college is part of a university, and a department is part of a college, 
consideration of centrality suggests concern for at least three levels: centrality to 
the mission and purposes for the larger university, centrality to the mission and 
purpose of the college, and centrality to the mission and purpose of a specific 
department. Centrality of activities at the program level might be considered an 
additional level or perhaps an alternative to considering the departmental level. 

While the criteria suggested here are thought to have application on several 
organizational levels, the primary focus of this discussion and multiattribute 
decision model is on review of departmental programs at the collegiate level. 
However, collegiate review must be in the context of the total institution (i.e., it is 
always possible that a program or activity might have lesser centrality for a 
college than it might have for the larger university or institutional framework). 
Does it contribute materially to programs of instruction foimd outside the college 
in other parts of the imiversity? When such cases of institutional centrality might 
exist, it is important that appropriate consultations occur among the various 
administrative levels, especially as regards decisions affecting program 
terminatiors. 

Perhaps the most direct indication of centrality is the degrjee of correspondence 
between program activity and the stated goals and missions of the college (the 
more specifically these are stated, the easier it is to establish the degree of 
correspondence). Most important, does the activity or program contribute directly 
to the research and instructional priorities of the college? If the activity were 
missing, would the college be compromised in carrjdng out its central mission? 
Another important indicator of centrality is the number of students from other 
units within the college who are enrolled in a program^s courses. However, 
caution should be used when reviewing such data. Such enrollments must be 
reflective of specific programmatic links with other areas of the college. One 
cannot, for example, have programs in teacher training without courses relating 
to learning theory, or programs at the graduate level without courses in 
statistical methodology. Thus, one form of centrality is the degree to which an 
activity or program provides specific programmatic services to other parts of the 
college. 

Yet another view of centrality is the degree to which the program or activity is 
related to the imique teaching and creative inquiry functions of the college as part 
of the university. Does it contribute materially to the interaction of instructional 
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programs with disciplined inquiry, of graduate level programs with initial 
licensure programs, or of discipline-oriented fields with problem and practice- 
oriented programs? All such questions must be considered as part of the concern 
about centrality in relation to mission. 

From these understandings about centrality to mission, the following 
measurable attributes of centrality were identified in our case study college: 

• • • Contribution of research to college mission 

• • • Contribution of instruction to college mission 

• • • Contribution of instruction to other tmiversity programs 

• • • Contribution of service to college mission 

P f^gyaiT^ f|p>mariH. A higher education institution is a basic resource and 
serves societal needs as an important part of its mission. Central to this mission, 
and one of the distinguishing characteristics of an American higher educational 
institution, is its primary role in providing instruction and training. Based on 
current enrollments, projected societal needs, and demographic trends one must 
try to predict likely future changes in program demand for presently existing and 
projected programs. 

Three aspects of program demand should be considered: (1) student demand 
for educational opportunity (i.e., individual student and social choice); (2) 
demand of the market for personnel (i.e., market and manpower forecasting); 
and (3) demand for ideas, information, and methods that may be developing 
through inquiry or testing (i.e., knowledge based social and professional 
requirements for research and development). These are obviously quite different 
demands and each should be viewed on its own merits. Their common element is 
that each judgment involves a comparison of a present program operation with 
an estimate of future student demand or expectations about future social needs. 

Information from department and program units regarding their perceived 
needs for new directions is one way to project student demand changes within 
and among programs. Such information, along with actual trends in programs 
and enrollments, should assist in the planning process of a college. For example, 
in considering programs often associated with colleges of education, increasing 
student and societal interest in early childhood education, educational 
technology, adult and community education, and continuing professional 
education clearly point to some directions for the future. A review of recent 
curricular changes concerning the addition or dropping of courses, along with 
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infrequently offered or low enrollment courses and program enrollment shifts 
over time, may also suggest student demand changes. Such information, 
combined with available demographic projections, should provide additional 
insight as to where an expansion of resources is likely to be needed or where 
reductions should be considered. 

It is important to note that quantitative measiires of student and social demand 
should not be the only guide for determining demand in a public professional 
college. These measures are useful for reviewing areas with either expanding or 
declining enrollments, but they have limitations. For example, if student 
enrollments have fallen off in some initial licensure program areas, this may not 
necessarily justify retrenching large portions of their budgets. The resource base 
may have been originally inadequate. Or, with programmatic shifts involving 
new clinical experiences or with an expansion into postbaccalaxireate licensure 
programs, smaller numbers of students may be essential, and additional 
resources even might be required. Although related to program quality, the ratio 
of qualified applicants to number of students admitted to a program also serves as 
evidence of student demand. Appropriate instructional ratios (e.g., full year 
equivalent students or student credit hours per full time equivalent staff, degrees 
per full time equivalent faculty, and the like), which might have different 
benchmarks for different areas and program levels, could be used as one type of 
information in developing guidelines for enrollment related resource needs. 
Other ways of operationally defining a "critical mass" of faculty and students 
(including acceptable floors and ceiling for student enrollments in individual 
programs) might also be developed for the different areas and programs across a 
college. 

With respect to the market (i.e. manpower) demand for personnel, one should 
review very carefully the history of placements for the program graduates 
wherein both the quality and proportion of such placements are examined. 
However, the most common form of projecting market demand for personnel in 
educational planning has been the use of manpower- forecasting techniques. In 
its simplest form, manpower-forecasting attempts to signal future labor market 
surpluses and shortages. When such models have been carefully developed with 
field-specific observations, they can provide useful guidelines as to the direction 
and intensity of expected changes. On the other hand, it is important to note that 
manpower-forecasting has generally proved to be a most elusive guide to social 
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and economic policy in both macro- and microeducation planning. The very real 
limits of such forecasting techniques have been frequently documented in the 
policy and planning literature (e.g., Psacharopoulos & Woodhall, 1985). 

From these descriptions of program demand, the following measurable 
attributes for demand were identified: 

• • • Current student enrollment 

• • • Projected social and student demand 

Uniqueness of prn yram^ The fundamental question being asked under this 
criterion is: What are the unique characteristics of this program that make it 
particularly appropriate to this university? Any argument for program 
imiqueness in a college is related to the social need (i.e^ demand) for such a 
program in the region, as well as the limited availability and accessibility of 
similar programs in other colleges within the state or region; however, the 
uniqueness of a program can and should be viewed as a separate criterion and 
dimension. Where such programs or aspects of programs (e.g., undergraduate 
instruction, inservice education, master level studies) are not geographically 
imique to a college and when they could be offered or are duplicated in other 
accessible colleges in the region, serious consideration should be given to the 
reallocation of such effort and resources. On the other hand, the possibility also 
exists that entirely new programs are needed (i.e., being demanded) and the 
comparative advantage of the university's mission and resources argue that it 
should be at the university. Periodically, each state supported college should 
review its curriculum and programs to consider whether there are programs 
offered elsewhere, but not in the region, that carry strong social need (i.e., 
demand) and are clearly within the department's or college's mission. Such 
programs should be considered for development. 

From these imderstandings about uniqueness of program, the following 
measurable attribute of program uniqueness was identified: 

• • • Uniqueness of program in state and region 

Cost^flFec^veness of pmgram. The extent of cost-effectiveness in any given 
program or activity is usually measured by comparing the costs or resources 
used with the outcomes or benefits achieved. It is a technique of selecting from 
alternative activities the one that will attain a given outcome at the lowest cost or 
the greatest benefits at given costs. The greater the benefits with given costs, or 
the lower the costs with given benefits, the greater the cost-effectiveness. It is 

14 

IG 



particiilarly useful where benefits caimot be measured in money terms. Such 
comparisons for determining cost-effectiveness are almost always expressed as 
ratios* The underlying notion in all such comparisons is that the use of 
resources involves a cost and that the outcomes should be compared with that 
cost. Thinking in **real" terms, the real cost of producing anything are the 
alternatives that are foregone. For example, the real cost of any given program or 
activity in a college would be the outcomes of the next best alternative program or 
activity given up that might otherwise have been produced with the same 
resources. Thus, for any resource allocation decision within a college, the extent 
of cost-effectiveness must be assessed by comparing the outcomes (at the margin) 
with the outcomes that might have been achieved if the same resources had been 
used in an alternative activity. To decide whether a department or a program 
should be maintained, expanded, or contracted, one must assess whether more 
desirable outcomes could be realized by reallocating resources into an area or out 
ofit. 

In discussion dealing with cost-effectiveness, three fallacies are frequently 
stated: (1) lower costs are better, (2) quality should not be related to cost, and (3) 
reduced costs mean reduced quality. Lowest or lower cost is not necessarily 
better. The key question is. What outcomes of what quality for what cost? Too 
often administrators fail to view the substantive differences in training methods 
and modes such as between professional and general education, between 
graduate and undergraduate education, and between a college with a relatively 
simple or imitary mission and a college in a multipurpose university with a 
miiltipurpose mission. Significant differences in quality of output are also 
frequently ignored. A second error is to judge cost-effectiveness only in relation to 
outcomes by assimiing that certain improved outcomes are desirable and should 
be sought irrespective of cost. The third error is the assumption that the quality of 
outcomes will necessarily fall if the costs for a given outcome are reduced. 
Comparisons of similar data among peer programs in other institutions often 
show this not to be the case. In higher education (as in any production process), 
there are a number of alternative ways in which contributing resources can 
function in order to produce outcomes. Some of these alternatives are obviously 
more cost-effective than others. The important idea is to recognize that cost- 
effectiveness involves a relationship between costs and outcomes. For example, to 
add to costs in a program area would not be cost-effective if the outcomes gained 
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were less valuable than the outcomes that the added resources would have yielded 
in another area. On the other hand, to cut costs would not be cost-effective if the 
outcomes given up were more valuable than the outcomes of the saved resources 
used in another area. 

It is important to note that departments and programs that operate at a low 
rate of expenditure per student are not necessarily cost-effective because of their 
limited resotirces. They may be cost-effective because of the way in which they 
manage their inputs (e.g., the consolidation of courses or sections with low 
enrollments) or because they do not require specialized training methods (e.g., 
large lecture classes as compared with individualized laboratory instruction), or 
because they have reoganized their curriculimi to reqtdre core courses for cohorts 
of students. Departments and programs that are cost-effective are not necessarily 
the lowest quality or lowest cost* In fact, examples of almost all combinations can 
be foxmd across most institutions. Again, the critical question tinderlying advice 
on cost-effectiveness is: How are program outcomes and costs related? For 
example, how can program outcomes be maintained (or enhanced) while 
reducing (or holding constant) costs? Or, what will be the tradeoffs in outcomes if 
resources are transferred from one program area to another? 

A major problem in all such assessments is how best to conceptualize and 
measure the inputs and outputs in order that they may be compared and 
evaluated. In higher education, inputs have often been thought of as dollar costs, 
although they might also be thought of as activity levels of faculty and staff (e.g., 

numbers of full time equivalent faculty and stafiD*^ Outcomes tend to be more 

complex and often more difficult to measure. Some of the simplest measures of 
output variables can be weighted (by level) student credit hours, student counts 
(full year equivalent or head counts), numbers of graduates, nxmibers of 
placements, numbers of publications, and numbers of external grants and 
contracts. Examples of more complex outcome variables include cognitive and 
affective changes in students, research and development contributions to 
professional practice, economic benefits (e.g., rates of return) to students and 
society, and other social benefits. The choice of which inputs and which outcomes 
will be used and how they will be defined, weighted and related are crucial 
determinations and should involve the most informed judgments of faculty and 
administrators. In a large university, useful comparisons can be made among 
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similar types of departments and programs within the college, as well as with 
other "peer** programs in other colleges and universities. 

Rather than seeking to establish an acceptable and comprehensive definition of 
program output for all units within a college (which is probably not possible), it 
seems best to make use of the concept of cost-effectiveness by determining selected 
indexes of effectiveness and by considering these indexes in relation to some 
measure of costs. 

From these imderstandings about cost-effectiveness, the following measurable 
attributes of oostreffectiveness were identified: 

• • • Average costs per student FYE 

• • • Average nximber of doctorates produced per year per PTE facility 

• • • Average nimiber of MEds produced per year per FTE faculty 

• ♦ ♦ Average nxmiber of BA/BSs produced per year per FTE faculty 

• # # FYE students per FTE faculty 

Agfiicfnincf Importance Weights to the Criteria and Attributes 

After both criteria and attributes have been identified and structured into an 
attribute tree (as illustrated in Table 1), the next step is to assign importance 
weights to both criteria and attributes. Not all of the criteria and attributes 
identified by stakeholders and/or by the decision maker are likely to be considered 
equally important; although this scenario would pose no threat to the overall 
evaluation. Most often, there are those criteria and attributes to any educational 
program which are considered more critical to the program's effectiveness than 
others. These differences in comparative value may also vary greatly from one 
individual stakeholder to another "Weighting" these variables is one way in 
which these comparative values can be analyzed. 

[Insert Table 1 about here] 
Although there are several different procedures for ranking criteria and 
assigning importance weights, the one most conventionally used is to assign each 
criterion with a value of relative importance on a scale of 0 to 100 and then to 
convert these values into proportion weights. This procedure is relatively easy to 
implement. A useful way to begin is to have the stakeholders first rank each of 
the criteria in terms of their relative importance. Then each stakeholder should 
assign a value of 100 to the most important criterion. Next stakeholders should 
assign a value between 0 and 100 to each of the remaining criteria that reflect their 
importance relative to the most important criterion. After these importance 
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weights have been established for each of the criteria, the same ranking 
procedure and assignment of importance weights should be imdertaken by each 
stakeholder with respect to each set of attributes identified within each of the 
criteria. 

This simple method offers some important advantages to other possible 
systems of weighting. One of the primary advantages is that is provides for the 
independent assessment of each individual criterion. Such freedom is not 
provided, for instance, in situations where the stakeholders may be forced to 
divide 100 points of importance among the differing criteria. This method of 
scoring importance weights also has the advantage that it is easily xmderstood 
and can be quickly implemented* This method also has the advantage of allowing 
for equal importance weights to be placed on the entire criteria set. If any 
stakeholder shoiild judge that all of the individual attributes for one of the criteria 
have equal importance, this should present no problem because the individual 
can simply assign 100 points to each of the items. 

Following this assignment of individual importance weights, the analyst 
should then convert the average scores of these weights into "proportion weights" 
by simply dividing each of the importance weights by the sum of the total values 
for the set of criteria or set of attributes. By definition, the total of all proportion 
weights for any set of criteria or attributes will equal 1.00. 

As each set of criteria and attributes are reviewed, the results of the individual 
importance rankings should be presented to the entire stakeholder group for 
further discussion. If any of the criteria or attributes evidence large variability in 
responses from stakeholders, another individual ranking exercise should be 
conducted for the set under review. It is anticipated that by doing this, the 
individual variability of rankings will be minimized through a group process of 
consensus building. The individual results of this second weighting should again 
be totaled and averaged to determine the final weights for the main criteria 
categories and attribute subcategories of the attribute tree. 

Although a common set of averaged weights (where each stakeholder has 
given similar weights to each criteria and attribute) with limited individual 
variability is the desired objective, this may not always be possible. In general, 
averaging is a useful technique when consensus is desired for the final valuing 

product.^ An alternative approach suggested by Newman and Edwards (1982) is 
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for the groups to essentially negotiate among themselves to arrive at an agreed 
upon set of weights. They correctly note that models using such negotiated 
weights are essentially political models of negotiated policy positions, jointly 
formtilated by multiple groups or individuals. Just as political and social policies 
are the result of group consensus, MAU models that reflect policy positions also 
result from group consensus- in this case, regarding the appropriate weights to 

use."* 

In Table 1 the importance weights for both the criteria and attributes of our 
case study have been identified. In viewing Table 1, the reader should note that 
the nimibers in the attribute tree are "proportion weights" for each of the criteria 
and attributes, and that they sum to one for all the criteria and for each subset of 
attributes. This is a conventional way of displaying the weights and in organizing 
for their subsequent use. In this procedure the final weights have been 
"normalized" (i.e., will sum to one), and assign to the most important and least 
important criteria and attributes the largest and smallest weighted numerical 
values respectively. 

The final weighted importance values for each of the individual attributes can 
now be calculated by a process defined as "multiplying through the tree." The 
reader should note fi*om Table 1, for example, that when attribute AA is 
multiplied through the tree it has a weighted importance value of .08. That is, 
when attribute AA's proportion weight of .26 is multiplied by its criterion (A) 
weight of .29, the product totals .08. It is this latter value which will be used in the 
technical analysis part of the evaluation when "weighted" utility values are 
estimated for each of the attributes foxmd within each of the program alternatives, 
Afifiiyniny Utilitv Values to the Attributes 

The next task is to assign utility values to each of our individual attributes 
through the construction of transformation graphs or utility functions. The 
transformation of measurable attributes to utility values is a straight forward 
procedure which can be explained either graphically or mathematically. 
Consider our case of evaluating resource allocation requests in higher education 
and assume that the stakeholders have determined that one of the attributes they 
wish to measure relating to program quality is the national ranking of the 
graduate program. 

[Insert Figure 1 about here] 
19 



The utility box in Figure 1 models graphically the transformation of this 
measurable dimension from quintile units of ranking scores into utility values. It 
illustrates, for example, that for each quintile from the bottom quintile of the 
national rankings, the program area would gain 25 units of utility. Knowing this 
exchange ratio permits the evaluator to then collect actual performance data from 
the alternative programs and compute the utility corresponding to the 
performance data. Other utility indexes and graphs can just as easily be 
constructed for other measurable attributes with other units of measure and 
scales, such as judgments about the quality of instruction and external funding, 
average costs per student FYE, and the like. 

There are two important points to remember. First, the transformation of 
differing scales and types of attributes into a standardized unit of measure is 
accomplished by the simple process of converting all raw score measures into 
utility values. Second, this transformation process is easily accomplished by 
establishing the exchange ratio or slope of the line in Figure 1. 

The usual transformation procedure for constructing the utility box and 
exchange ratio illustrated in Figure 1 is to first examine the likely range of 
expected performance on the attribute with maximxim and minimum values; and 
then establish this range as the relevant range for estimating the zero and 100 end 
points on the utility scale. Graphically, a single 45 degree line can then be drawn 
within the utility box to approximate the utility transformation and functional 
relationship. In the case of Figure 1, the slope of the 45 degree line expresses the 
exchange rate between quintiles of national rankings and units of utility (i.e., 
value or satisfaction). This exchange ratio is simply the slope coefficient of a 
straight line and also can be expressed in mathematical form. 

The utility relationships of this case study all assume a linear relationship 
throughout the relevant ranges. This linear relationship is represented by the 
straight 45 degree line in the utility box of Figiire 1. Other curvilinear 
relationships can just as easily be constructed and in some cases may be better 
representations as to how some people might judge rates of satisfaction (i.e., 
utility). However, research in the field (Edwards, 1980; Keeney, 1977) has indicated 
that such curvilinear representations almost never make any difference to the 
decision outcome largely because of the small differences in utility values which 
might result. Nevertheless, if linear relationships for certain attributes prove 
troublesome to any decisionmakers or stakeholders, it is a very simple process to 
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create a utility box with curvilinear transformation lines. The evaluator can 
simply draw a graph similar to Figure 1 with the measurable dimension of the 
attribute on the X axis and the 0 to 100 point utility scale on the Y axis, and then 
draw whatever functional curve most appeals to the stakeholder or group. 

The task of assigning utility scales to each of the attributes in a MAU 
evaluation is usually undertaken by the evaluator, in consultation with 
knowledgeable individuals in the field for making estimates about the plausible 
ranges of attribute performance. This results because the determination of 
ranges is lairgely an empirical and technical question. However, in higher 
education once plausible performance ranges have been tentatively established, it 
is almost always a useful strategy to review these ranges and the shape of the 
utility functions with the principle stakeholder(s). When emplojdng an MAU 
evaluation in higher education, the involvement of facility stakeholders in this 
process is important primarily because of the significant political role faculty play 
in both the development and execution of college policy. The validity and 
utilization of the entire evaluation will be largely determined by the acceptance 
and support given to the process of the evaluation by participating stakeholders. 
When they have a role in determining not just the identification of attributes and 
assignment of importance weights, but also a role in confirming utility values, 
their support and the validity of outcomes will be enhanced. 
CoBectmg pprfnrmance Dato from/for the RpiJlocation Rpnuests 

After our MAU evaluation model was framed and assigned importance 
weights and utility scales for each of the attributes, we collected data from and for 
each of the 16 reallocation requests. In the assessment of programs in higher 
education, it is expected that such attribute performance data will come fi-om one 
of three sources of information. Some data will be collected from the college's 
internal records that include information on such items as costs, teaching loads, 
degrees awarded, external funding, and the like. Other data will be drawn from 
surveying the judgment(s) of the decision maker or stakeholders on such issues 
as the uniqueness of the program in the state or region. However, most of the data 
needed for scaling the attributes should come directly from the reallocation 
requests themselves, particularly when the submitting program areas 
understand and know that their requests will be judged against certain specific 
criteria with associated attributes. The instrumentation for collecting this 
information in the case study can be viewed in Table 2 wherein scales are also 
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identified for each of the attributes. 

[Insert Table 2 about here] 
Conducting th^ MATT Analvgis 

Through the procedures of the previous steps in developing oxir MAU 
evaluation model, we constructed three sets of numbers for each of the attributes 
in our evaluation model: (1) weighted importance values, normalized to sum to 
1.0; (2) utility values assigned to each imit of measured performance, expressed on 
a utility scale of satisfaction from 0 to 100; and (3) adjusted performance measures 
for each alternative decision package, wherein raw performance scores are 
adjusted for lowest plausible performance level. The next step in our MAU 
procedure was to aggregate these numbers into a weighted utility score for each 
attribute for each alternative request, and then sum all of these weighted utility 
scores for each alternative. 

The assignment of weighted utility to each attribute is a simple process which 
requires only multiplying the weighted importance assigned to each attribute 
times its assigned utility unit value times its adjusted performance measure for 
each alternative. Total weighted utility for each resource reallocation request is 
then derived by just adding up the products for each of the attributes. This 
procedure is illustrated in Table 3 wherein each of the attributes are listed along 
with their assigned normalized importance weights (column 1) and constructed 
unit utility values (column 2) common to all the alternatives. Also included in 
Table 3 are the adjusted measured performance (colimin 3) on each attribute from 
each alternative. It is important to note that these performance measures are 
adjusted according to whether "more is better" or 'less is better" as the desired 
performance value. When more is better on the performance scale, the 
adjustment is to subtract the minimum plausible score from the actual 
performance number. When less is preferred as a performance outcome, the 
adjustment is to subtract the actual performance number from the maximum 
plausible score. Each of these three values (i.e., the normalized importance 
weight, the attribute transformation ratio, and the adjusted performance score for 
the ith attribute) are then multiplied through the table to arrive at a weighted 
utility value (column 4) for each of the attributes for each of the alternatives. 

[Insert Table 3 about here] 

The weighted utility values for each of the attributes are then added up for each 
of the alternatives to arrive at a total weighted utility value for each alternative. 
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Because of the methodology used in this MAU evaluation, the larger the 
numerical value of each total weighted utility score, the "better" or "more 
effective" that alternative program can be considered by the evaluator or final 
decisionmaker. This total weighted utility value of each alternative can be now 
used in making comparisons between alternative programs, or between differing 
timespans of the same program. With the values reported in Table 3, for example, 
we can rank the reallocation request for the Alternative #9 program as having the 
highest priority for reallocation consideration among the four alternatives 
illustrated. 

Table 3 also illustrates several distinguishing characteristics relative to 
program performance (i.e., colimm 3) across four of our case study allocation 
requests. Note that the performance profiles for our top two alternatives #9 and 
#1 give clear evidence that both programs have outstanding quality, strong 
centrality to college mission, adequate student demand and uniqueness, and are 
highly cost-effective. Alternative #2, on the other hand, looses utility value 
because of its very weak quality while holding strong in centrality, student 
demand, and cost-effectiveness. Finally, alternative #3 is generally weak across 
all criteria, except that it appears to be relatively cost-effective in its instructional 
activities. 

Table 4 reports on the relative ranking of all 16 of the alternative program 
requests. Although there was considerable variability across the 16 alternatives 
on their adjusted performance on individual attributes, this variability was 
somewhat muted through the assignment of importance weights. This resulted 
in less variability in the total weighted utility scores for some of the alternatives. 
Nevertheless, the final ranking of effectiveness scores (i.e., as measured through 
weighted utility scores) gives clear indication that several of the alternative 
programs are at least four to five times more effective across all of the attributes 
than the lowest alternatives. We note from Table 4 that the three highest utility 
values were assigned to allocation request numbers 9, 1, and 16 with values of 81, 
76, and 74 respectively. On the other hand, at least five of the allocation requests 
were ranked as clearly on the bottom (i.e., allocation request numbers 8, 13, 11, 10, 
and 3) with weighted utility values of only 30, 30, 26, 17, and 16. 

[Insert Table 4 about here] 
The ranking of all 16 of the allocation requests in Table 4 gives clear evidence 
that MAU modeling and evaluation procedures can provide meaningful scaling of 
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the perceived worth and effectiveness for resource allocation decision making. It 
is clear that three of the programs have the strongest case for preservation and 
funding, another group of four to eight programs have some strength but 
apparent weaknesses as well, while the five lowest ranked requests should signal 
them as clear candidates for retrenchment. 

rhP^rking tb*> Final Rftsiilte for Sensi* iViHr nf Analvfiis tn Assigned Weights and 
JiiHgtnents 

In order to test the results in our case study to determine whether the 
assignment of an additional criteria, differing importance weights, or differing 
performance judgments might make any material differences in our final 
ranking results reported in Table 4, we conducted several sensitivity tests with 
different numbers. In none of these additional tests were the final results 
materially changed. For example, the additional criteria of "uniqueness" that 
was adopted by the all-university planning process was removed from the 
attribute tree (i.e., assigned an importance value of zero) and all other parameter 
and performance values were retained. In this case, none of the top three 
rankings were changed and the bottom five rankings remained at the bottom. 
Only some of the intermediate rankings changed places but only by one to three 
unit changes in weighted utility. Similarly, when importance weights were 
modestly changed by entering the independent judgments of a new assistant dean 
who did not participate in the original MAU exercise, the final rankings did not 
materially change. Other independent judgments on several of the performance 
scales were also entered as a separate simulation to test for the sensitivity of this 
dimension, and no material changes in either the final ranking or the variance 
between requests were observed, 
fiiimmarv 

Data on the performance of 17 identified measurable attributes found within 
five criteria were collected from a two-year sample of 16 alternative reallocation 
requests within a large college of education. These performance data were then 
converted into utiUty measures (see Figure 1) and loaded into an attribute value 
tree with assigned importance weights for determining the final utiUty weights 
for each of the 16 resoiu-ce requests (see Table 3). The total weighted utiUty values 
in Tables 3 and 4 indicate the relative effectiveness (i.e., expected value) of the 16 
alternatives based on their respective performances. From the results reported in 
T^ble 4 it can be seen that the 16 alternatives can be ranked from the most effective 
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and highest valued option with a weighted utility value of 81, to the lowest valued 
request with a weighted utility of 16. 

The use of MAU evaluation techniques in assisting decisionmakers in higher 
education for making resource allocation decisions about retrenchment and 
reallocation is clearly invaluable. It not only adds validity and rationality, but it 
adds an orderly structure and process as well. However, the cards will not always 
read themselves. Inherent in all major resource decisions in higher education is 
the need to be sensitive to extenuating circumstances that are not always 
apparent in the data. For example, programs for which there is high demand 
and are highly cost-effective, but that tend to be marginal with respect to other 
criteria, are especially bothersome. Such programs must be reviewed very 
carefully so as not to detract from other more central programs and from the 
primary mission of a college. Support decisions are also difficult when highly 
central programs are of generally low quality combined with limited demand and 
high cost. Extra effort might need to be directed to improving the quality and cost- 
effectiveness of such programs. Similar problems arise when a program exhibits 
generally high quality but is viewed as being low with respect to demand or 
centrality. Nevertheless, the overall prograxn profile that emerges from 
considering all elements and their relationships as in an MAU type analysis will 
make a useful contribution to the complicated and judgmental resource allocation 
process of most colleges. 
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Endnotes 



The original list from the combined two years included 20 total allocation 
requests. However, four of the requests were duplicates over the two years and 
were accordingly removed from the combined number of alternatives for this 
case study. 

The advantage of defining input in dollar terms is to establish a readily 
available measure of input that is comparable across programs. In no sense 
should this conceptualization of inputs be assimied to mean that budgeted 
finances are the only important inputs to higher education. In some 
instances dollars may be a poor surrogate or indirect measure of the true 
resources that contribute to outcomes. 

However, consensus (or near consensus with limited variability in individual 
responses) is not always possible. When individuals or groups of stakeholders 
disagree on importance weights, it is usually a good idea to discuss the 
disagreement and attempt to resolve whatever the issue might be. On the 
other hand, when such disagreement is fundamental, it is best to save both 
sets of weights for later analysis. Fundamental differences in importance 
weights can be either treated as a separate evaluation or they can be addressed 
with subsequent sensitivity analysis. In sensitivity analysis with MAU 
models, where the alternative weights are placed into a separate analysis, the 
literature indicates (see Newman, 1977; Huber, 1980) that the final results are 
often insensitive to whichever set of weights are used. Nevertheless, it is 
essential that such separate treatment of material differences in opinions be 
preserved in order to test for possible important differences in the final results, 
and to insure the face validity and integrity of the process with the stakeholder 
groups. 

There are several other techniques for establishing rank ordering and 
weighting of listed items, including rank sum weighting and rank reciprocal 
weighting; however, all rank weighting methods in MAU analyses are at best 
approximations. Our ranking procedure is recommended largely on the bases 
of its simplicity and ease in use. For the reader interested in reviewing these 
other ranking techniques, an excellent source can be found in Van Winterfeldt 
and Edwards (1986). 
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Table 1: Value Tree of Criteria and Measurable Attributes Showing Proportional Weights 
and Final importance Weights 



Proportional Weighted 
Weights Importance 



A 0.29 Quality of Outcomes 

AA 0.26 Graduate program national ranking 0.08 

AB 0.25 Quality and rate of faculty publications 0.07 

AG 0.16 Quality and rate of external funding 0.05 

AD 0.23 Quality of instruction 0.07 

AE 0.1 0 Quality of professional service 0.03 

B 0.23 Centrality to Mission 

BA 0.31 Contribution of research to college mission 0.07 

BB 0.31 Contribution of instruction to college mission 0.07 

BC 0.16 Contribution of Instruction to other university programs 0.04 

BD 0.22 Contribution of service to college mission 0.05 

C 0.14 Demand for Program 

CA 0.47 Current student enrollments 0.07 

CB 0.53 Projected social and student demand 0.07 

D 0.06 Uniqueness of Program 

DA 1 .00 Uniqueness In state and region 0.06 

E 0.28 Cost-Effectiveness of Program 

EA 0.25 Average costs per FYE student 0.07 

BB 0.25 Average doctorate per PTE faculty 0.07 

K 0.1 3 Average MEd per PTE faculty 0.04 

a) 0.12 Average BA/BS per PTE faculty 0.03 

EE 0.25 PYE students per PTE faculty 0.07 

Total 1.00 



3i 



Table 2: RATING OF INDIVIDUAL ALLOCATION REQUESTS 
ON SCALES OF MEASURABLE ATTRIBUTES 



A. QUALITY: 

AA) Graduate program national ranking 

1) . 2) 3) , 4) , 5) 

unranked top 20-40 top 10-20 top 5-10 top 5 

AB) Quality of faculty publications 

1) . 2) . 3) 

low medium high 

AC) Quality of external funding 

1) . 2) . 3) 

low medium high 

AD) Quality of instruction 

1) . 2) . 3) 

low medium high 

AE) Quality of professional service 

1) . 2) . 3) 

low medium high 

B. CENTRALITY 

BA) Contribution of research to college mission 
1) . 2) . 3) 

low medium high 

BB) Contribution of instruction to college mission 
1) 2) . 3) 

low medium high 

BC) Contribution of instruction to other university programs 
1) . 2) . 3) 

low medium high 

BD) Contribution of service to college mission 
1) 2) 3) 

low medium high 



ERIC 
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C: DEMAND 

CA) Current student enrollments (i.e., average tenure track faculty SCH) 
1) . 2) 4) 4) 

4thquartile 3nclquartile 2rd quartile topquartile 

CB) Projected social and student demand 
1) 2) , 3) 

low medium high 

D. UNIQUENESS 

DA) Uniqueness of program in state and region 

1) 2) , 3) 

low medium high 

E. COST-EFFECTIVENESS 

EA) Average costs per student fVE 

1) 2) 3) 4) 5) 

over $8000 $7-8000 $6-7000 $5-6000 under $5000 

EB) Average number of doctorates produced per year per tenure track FTE faculty 
1) . 2) 3) 4) 5) 

0-.30 .30-.80 .80-1.50 1.50-2.00 over 2 

EC) Average number of MEds produced per year per tenure track FTE faculty 

1) . 2) , 3) 4) 5) 

0-.30 .30-1.00 1.00-2.00 2.00-4.00 over 4 

ED) Average number of BA/BSs produced per year per tenure track FTE faculty 
1) 2) , 3) 4) , 5) 

0-.50 .50-2.00 2.00-4.00 4.00-10 over 10 

EE) FYE students per tenure track FTE faculty 

1) , 2) , 3) 4) 5) 

0-5 5-8 8-12 12-16 over 16 
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Table 4: Ranking of Allocation Requests 



Department/Program Ranking by 

Allocation Request Package Total Weighted Utility 



#9 


81 


#1 


76 


#1 6 


74 


#15 


61 


#4 


57 


#14 


54 


#2 


51 


#12 


48 


#6 


48 


#5 


46 


#7 


44 


#8 


30 


#13 


30 


#11 


26 


#10 


17 


#3 


16 
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Figure 1 : Transformation of Measurable 
Attribute into Utility Values 
(Utility Box for Attribute AA) 




